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Abstract 


Nearly two decades into the “completion agenda” in higher education, many 
community colleges have adopted collegewide reforms designed to improve stubbornly 
flat rates of student success and address persistent equity gaps. The longer-term effects of 
such collegewide reforms may take years to observe. In the meantime, college leaders 
need to know whether changes they make in the short run are associated with longer-term 
student success. Measuring the progress and effects of institutional reform is particularly 
vital in economically important STEM fields. Drawing on administrative records from 
transfer-intending community college starters across three states, this study develops and 
explores potential indicators of early STEM program momentum. We find that a 
relatively simple set of STEM momentum metrics—notably early completion of calculus 
or non-math STE coursework specified in statewide STEM transfer pathways and, to a 
lesser degree, the prerequisites to such courses—are reliable indicators of subsequent 
STEM transfer and bachelor’s degree attainment. Our findings provide support for the 
use of the STEM momentum metrics to formatively evaluate reforms aimed at 
strengthening STEM transfer outcomes and closing equity gaps in STEM bachelor’s 


degree attainment. 
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1. Introduction 


Nearly two decades into the “completion agenda” in higher education, many 
community colleges have adopted collegewide reforms designed to improve stubbornly 
flat rates of student success and address persistent equity gaps (Bailey et al., 2015; 
Jenkins et al., 2020). The longer-term effects of such collegewide reforms may take years 
to observe. In the meantime, college leaders need to know whether changes they make in 
the short run are associated with longer-term student success. Accordingly, college 
leaders have turned to “leading indicators” or “early momentum” measures, which are 
meant to provide timely, formative feedback to support continuous improvement as 
collegewide reforms are implemented and scaled up (Jenkins & Bailey, 2017). 

At the institutional level, metrics of early momentum are program-agnostic. For 
example, the widely used metric of first-year credit accumulation carries an intuitive and 
consistent meaning, regardless of whether students are aiming for a degree in business, 
engineering, education, or a variety of other fields. Yet on the level of an individual 
student, program-specific coursetaking may be the strongest indicator of eventual transfer 
or graduation (Denley, 2016; Jenkins & Cho, 2012, 2014). Moreover, institutional 
improvement aimed at greater student success requires the involvement of faculty and 
administrators within specific academic programs, who may be better equipped to engage 
in reform efforts if metrics are contextualized to their own programs (Bailey et al., 2015). 
While Wang (2017) has articulated its theoretical importance, thus far the field lacks a 
framework for developing and deploying reliably predictive and practical program- 
specific momentum metrics. 

Measuring the progress and effects of institutional reform is particularly vital in 
the economically important fields of Science, Technology, Engineering, and Math 
(STEM). Community colleges represent an important potential pipeline for diverse talent 
into STEM fields, which have struggled even more than other fields to increase 
representation of Black and Hispanic workers, women, and individuals from other 
minoritized groups (Riegle-Crumb et al., 2019). More than a million students enter higher 
education through community colleges each year, including large proportions of Black, 
Hispanic, low-income, and first-generation students (American Association of 


Community Colleges, 2021). Yet few community college students progress to advanced 


STEM coursework and transfer into a STEM bachelor’s program. For example, of 1.7 
million students nationwide who started higher education at a community college in the 
2007-08 academic year, only about 3% ever transferred and completed a STEM 
bachelor’s degree (Jaggars et al., 2016). Underlying these low completion rates are 
challenges resulting from ineffective institutional practices and misaligned transfer 
systems, which have disproportionately impacted STEM-aspiring women, students of 
color, and other marginalized groups (Wang, 2020). Despite its potential for broadening 
access to STEM bachelor’s degrees, the STEM transfer pathway is underperforming and 
inequitable. 

This study develops and explores potential indicators of early program 
momentum, with a focus on transfer pathways to baccalaureate degrees in STEM. We 
draw on administrative records from nearly 270,000 transfer-intending students who 
began at 70 community colleges across three state systems and, as a basis for comparison, 
nearly 57,000 students who began at 26 broad-access, regional four-year campuses in the 
same states. We leverage these data to address the following research questions: 

1. Cana simple set of STEM momentum metrics predict 
students’ long-term transfer outcomes at a similar or 


superior level as widely used general early momentum 
metrics? 


2. Are these STEM momentum metrics reliable across a wide 
variety of institutional contexts and student groups, 
particularly those students who are historically 
underrepresented in STEM? 


3. To what extent do these metrics reflect students’ intent to 
study STEM, students’ success within STEM, and 
institution-specific efforts to support STEM pathways? 

Our findings suggest that STEM-specific coursetaking measures are just as or 
more predictive of subsequent STEM transfer outcomes than general early momentum 
measures and that the most consistently useful indicator is first-year completion of STEM 
courses specified on statewide STEM transfer pathways. However, relatively few 
community college students complete this type of STEM coursework in their first year; 


students are instead more likely to concentrate on completing prerequisite “foundational” 


STEM courses or other STEM courses that may transfer to a four-year college without 
fulfilling STEM bachelor’s degree requirements. Despite the potential of community 
college STEM transfer pathways to broaden access to STEM careers for 
underrepresented groups such as Black and Hispanic students and women, we find low 
and inequitable rates of early STEM transfer momentum. Encouragingly, however, we 
find that early completion of key STEM transfer courses is a reliable predictor of 
subsequent STEM transfer success across student demographic subgroups and 
institutional contexts. Taken together, our findings suggest that a promising approach 
toward increasing STEM bachelor’s attainment and closing equity gaps in STEM fields is 
to focus on helping first-year students complete specific courses designated on their 
state’s STEM transfer pathways. 

In the remainder of this introduction, we first situate our study within the context 
of state- and institutional-level improvement efforts to support community college 
student success in STEM and other fields. We then introduce the use of early momentum 
metrics as leading indicators of collegewide reforms, highlighting the need for better 
measures of program momentum. We turn to prior empirical and theoretical literature, as 
well as related work on STEM transfer momentum specifically, to provide conceptual 


grounding for this study. 


1.1 State and Institutional Community College Improvement Efforts 

Across the country, community colleges are redesigning programs and support 
services to improve student success in STEM and other fields. To do so, many have 
adopted the framework known as guided pathways, an ambitious, whole-institution 
transformational approach designed to improve how students enter and navigate through 
programs of study to earn credentials (Bailey et al., 2015). Under the guided pathways 
framework, colleges engage in the fundamental redesign of programs, practices, and 
systems across four areas (Bailey et al., 2015). First, colleges organize programs into 
career-focused “meta-majors” (e.g., health sciences, computers and information 
technology) and work with employers and university partners to clearly map every 
program to job and transfer destinations. Second, colleges redesign the student 
onboarding experience to accomplish two key goals: (1) help all new students explore 


and understand program and career options and develop a full program plan, and (2) 
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integrate academic support and evidence-based teaching methods into critical 
introductory courses. Third, colleges empower advisors to monitor students’ progress on 
their plans, provide frequent feedback, and intervene when students need help or want to 
change course. And fourth, colleges align program learning outcomes with the 
requirements for success in that program’s job and transfer destinations and integrate 
active and experiential learning throughout programs to engage students in learning that 
is relevant to their interests and goals. Guided pathways reforms are underway in more 
than 400 colleges across the country, and, in at least 16 states, these efforts are 
incentivized by or coordinated as part of state-level reform efforts (Jenkins et al., 2019, 
2020; Community College Research Center, 2021). 

Preparing students for transfer to a four-year institution is a core mission of many 
community colleges, and colleges implementing the guided pathways framework have 
aimed to strengthen transfer outcomes. Among other reforms, colleges have worked on 
increasing the transferability of community college students’ credits to four-year 
institutions, a key predictor of transfer students’ likelihood of completing a bachelor’s 
degree (Monaghan & Attewell, 2015). Indeed, over the past two decades, most states 
have newly implemented or further developed some type of statewide transfer agreement, 
which could include a common set of general education requirements across public four- 
year colleges, a common course numbering system, or a guaranteed transfer policy that 
automatically admits students into a destination college contingent upon completion of an 
associate degree (Kisker et al., 2011; Roksa, 2009; Zaragoza, 2021). While research on 
states’ transfer improvement efforts has been limited, it suggests that early efforts did not 
noticeably improve transfer rates (Anderson et al., 2006; Gross & Goldhaber, 2009; 
Roksa, 2009). Additional reforms are sorely needed. While about 80% of new, degree- 
seeking community college entrants nationwide intend to transfer to a four-year college, 
only about 30% transfer within six years, and fewer than 15% complete a bachelor’s 
degree (Shapiro et al., 2017). And this already underperforming transfer system 
disproportionately affects Black, Hispanic, and low-income community college students 
(Shapiro et al., 2017, 2019). 

The guided pathways framework suggests that past efforts to improve community 


college transfer rates may have been hindered by their focus on general education 


coursework. The conventional wisdom that transfer students should “get their gen eds out 
of the way” at community college may be flawed because not all transferable gen eds 
fulfill bachelor’s degree program requirements. Fink et al. (2018) highlighted this 
distinction between credit transferability and applicability, identifying widespread 
misalignment of early community college coursework with bachelor’s degree 
requirements even among successful community college transfer students who completed 
a bachelor’s degree. For example, in two state systems, community college transfer 
students who completed a bachelor’s degree graduated with more excess credits above 
bachelor’s degree requirements if they enrolled in higher proportions of introductory 
coursework at community college and if they had to retake key courses such as college- 
level math after they transferred. Misalignment of community college transfer 
coursework with bachelor’s degree requirements and the resulting inefficiency in credit 
transfer have been not only documented as penalizing successful transfers in terms of 
excess credits and time to completion (Cullinane, 2014; Xu et al., 2017) but also shown 
to be a key factor in predicting whether community college transfer students will 
complete a bachelor’s degree in the first place (Monaghan & Attewell, 2015). 

In contrast to a focus on general education coursework, the guided pathways 
framework suggests that transfer efforts should focus on helping students explore and 
connect to fields of interest upon entry at community college. In accordance with the 
guided pathways framework, more recent community college reforms focus on aligning 
early coursetaking to baccalaureate programs to ensure students’ credits apply to four- 
year degree requirements (Zaragoza, 2021). In particular, some states have recently 
moved toward program-level transfer agreements, which apply and expand existing 
statewide policies in the context of specific programs, such as business, education, or 
chemistry. Program-level transfer agreements clearly articulate pre-major and transfer- 
level program requirements agreed upon by institutional leaders across the state, 
providing students with a roadmap to enter directly into participating four-year 
institutions with junior standing in their chosen major. Such program-specific agreements 
can reduce the accrual of unnecessary or excess credits and increase the likelihood of 
transfer students completing a degree (Baker, 2016; Fink et al., 2018; Fink & Jenkins, 


2020). By identifying common, program-specific foundational courses for fields of study 


across an entire state, community colleges and their primary four-year partners are better 
equipped to advise students on which early courses will not only transfer but also apply 


to their bachelor’s degree program (Wyner et al., 2016). 


1.2 Early Momentum as a Metric for Formative Evaluation 

The growing guided pathways reform movement is increasing the demand for 
practical and timely metrics that colleges can use to understand whether their reforms are 
on the right track or need redirection. Guided pathways reforms are difficult to evaluate 
because guided pathways is a whole-college reform model intended to “treat” all students 
at a college; thus, reformers do not have a clear control group for comparison. Moreover, 
colleges implementing new reforms cannot afford to wait three years or longer to observe 
changes in student graduation rates; they need actionable information as early as possible. 
Thus, many colleges implementing guided pathways have turned to early momentum 
metrics: simple measures such as student enrollment intensity, continuity, and first-year 
credit accumulation, which are good predictors of associate degree completion, transfer 
to a four-year college, and bachelor’s degree completion (Attewell & Monaghan, 2016; 
Belfield et al., 2016, 2019; Wang, 2017). Crucially, these metrics are measurable after 
only one year, and institutions can evaluate their progress by observing year-over-year 
trends. Early momentum metrics are key tools in formative assessment of reforms; they 
not only allow leaders to determine if reforms are resulting in improvements for students 
generally but also can signal progress toward closing equity gaps if institutions 
disaggregate metrics across key student subpopulations (Belfield et al., 2019). The 
predictive power of early academic milestones has been shown to be especially strong for 
Black, Hispanic, and low-income students (Lin et al., 2020), suggesting that leaders 
working to close equity gaps in longer-term outcomes should focus their efforts on 
closing such gaps in the near term. 

While early momentum metrics are useful, they do not capture program 
momentum, or the extent to which students are making progress in their chosen program 
of study. Wang (2017) has mentioned program momentum as theoretically important to 
the concept of momentum, but only three empirical studies have explored it. Two studies 
examined whether community college students entered a program of study within the first 


year of enrollment, which researchers define as earning nine credits or completing three 
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courses within any specific program area. Jenkins and Cho (2012) showed that 
community college students who failed to meet this bar were much less likely to transfer 
and earn a credential. Denley (2016) showed that students in both community colleges 
and public universities who met this bar were more than twice as likely to complete a 
college credential in six years than students who did not meet this bar. To measure 
STEM-specific momentum, Wang (2015) created a measure of first-term “STEM quality 
points” for community college students, which combines the number of credits 
accumulated and grades earned in any type of STEM course, finding that these points 
predict STEM bachelor’s degree completion among students intending to transfer into 
STEM. 

From a practical perspective, program momentum may be a more helpful 
construct than general early momentum for two reasons. First, qualitative researchers 
have identified students’ program of study as a promising focal point for student success 
reform. In particular, community college programs that have more clearly defined 
program requirements, stronger program coherence, and integrated academic and 
nonacademic supports also have substantially stronger performance in terms of student 
credential attainment and rates of transfer to four-year college (Carey, 2008; Jenkins & 
Cho, 2014; Scrivener et al., 2012). Second, student success improvement efforts, 
program-specific or otherwise, are not feasible without the engagement and commitment 
of program faculty. Yet for faculty who work within specific departments or programs, 
general institutional metrics may seem abstract or irrelevant to their daily work; in 
contrast, program- or field-specific metrics may be more meaningful and engaging 
(Bailey et al., 2015). Thus, inclusion of program-specific measures in addition to general 
early momentum metrics creates more opportunity for engagement of program faculty in 


institutional reform efforts. 


1.3 STEM Transfer Program Momentum 

Improvements in program momentum and completion are of particular concern in 
the area of STEM transfer. STEM workers are in demand by employers and earn high 
wages in the labor market, but well-paying and fast-growing jobs in STEM typically 
require a bachelor’s degree (Fayer et al., 2017). Nationally, it is unclear how many STEM 


bachelor’s degree earners start at a community college and follow a transfer pathway. 
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However, among bachelor’s degree graduates overall, nearly half attended a community 
college at some point, including those who took occasional summer courses or pursued 
dual enrollment in a community college during high school (National Student 
Clearinghouse Research Center, 2017). This proportion may be even higher among 
female, Black, and Hispanic STEM degree holders (Mooney & Foley, 2011; Tsapogas, 
2004). 

Compared to similar four-year college starters, community college starters are 
less likely to successfully earn a bachelor’s degree in STEM (Hu & Ortagus, 2019; 
Monaghan & Attewell, 2015; Wang, 2015). Many STEM programs at community 
colleges are marked by an absence of widely accepted sequences of transferable courses, 
and major-specific program maps may be nonexistent or infeasible for students to follow, 
due to a misalignment between the early coursework that some four-year STEM 
programs demand and the coursework that most community colleges can offer (Bahr et 
al, 2017; Jaggars et al, 2016). Community college students from relatively privileged 
backgrounds are more likely to have the financial and social network resources necessary 
to navigate through such barriers, while traditionally underserved students are more likely 
to suffer from them (Jabbar et al, 2019; Jaggars et al., 2016; Wang, 2020). However, 
compared to four-year colleges, community colleges often provide a climate that is more 
welcoming to women and students of color and thus have the potential to close gaps in 
STEM achievement for these populations (Hu & Ortagus, 2019; Jackson & Laanan, 
2011; Mooney & Foley, 2011; Tsapogas, 2004). 

To improve transfer pathways and student performance in STEM, community 
colleges need measures of momentum that are relevant to these pathways. General early 
momentum metrics, or even metrics based on general STEM coursetaking, may not be 
highly relevant to STEM transfer pathways because not all courses within a given 
program apply to bachelor’s degrees in the same area (Bailey et al., 2015; Fink et al., 
2018; Lyon & Denner, 2016). (For example, many students take “non-major” versions of 
biology, chemistry, physics, or other STEM courses as part of their general education 
requirements, but these courses are not designed to fulfill major-specific requirements for 


bachelor’s degrees in the same fields.) 


To measure STEM program momentum, researchers need clear definitions of 
STEM versus non-STEM courses, an understanding of which STEM courses will transfer 
to a four-year college, and an understanding of which STEM courses will actually apply 
toward earning a STEM bachelor’s degree. The work of Wang (2016) and Chan and 
Wang (2020) provide a strong foundation in this regard. First, Wang (2016) used two- 
digit CIP codes to identify STEM courses and classify those that are “likely transferable” 
to a four-year college. The limitations of this system of classification are that, even within 
the category of likely transferable STEM courses, some courses may not actually transfer, 
grant a student access to a STEM major post-transfer, or apply to the student’s STEM 
bachelor’s degree program. Thus, Chan and Wang (2020) further refined the taxonomy 
by drawing on institutions’ transfer articulation agreements to identify courses that are 
typically transferable and classify them as either foundational (e.g., general physics, 
general geology) or advanced (e.g., organic chemistry, microbiology). 

Using the refined taxonomy, Chan and Wang (2020) found a modest alignment 
between students’ transfer intent and their coursetaking patterns and likelihood of 
completion. For instance, community college students interested in pursuing a bachelor’s 
degree in a STEM field were more likely to take transferable STEM courses, which in 
turn improved their likelihood of transfer as well as degree attainment. Wang (2016) used 
transcript data analysis to show that community college students who ended up being 
successful in STEM transfer tended to earn more credits in likely transferable STEM 
courses and fewer credits within other non-STEM categories both in their first year and 
throughout the six-year window of the study. Yet this approach to identifying transferable 
STEM courses is incomplete in the context of GP-aligned state and institutional reform 
efforts, which aim to ensure that courses not only transfer but also apply to each student’s 
bachelor’s degree program. 

In this paper, we build on prior empirical work in the context of three states in 
which transfer reforms have led to the creation of statewide STEM-specific transfer 
pathways. In identifying new measures of STEM transfer momentum, we extend Wang’s 
(2016) concept of likely transferable STEM courses to those that are actually specified on 
statewide STEM transfer pathways as satisfying STEM baccalaureate degree 


requirements. In doing so, we advance scholarship around early STEM momentum while 


also shedding new light on the potential of statewide transfer reforms to improve credit 
transfer efficiency and transfer student success. We identify and test a set of potential 
STEM transfer momentum metrics and then compare these against well-established early 
general academic momentum metrics, concluding with an examination of their usefulness 


across different types of students and an exploration of their implications for practice. 


2. Methodology 


2.1 Examining STEM Transfer Pathways in the State Context 

Our study focuses on three anonymized states, A, B, and C, that have made 
progress in implementing program transfer agreements using guided pathways principles. 
The three states are midsize with between 10 and 40 community colleges in their state 
system, and each was willing to share its centralized student unit record data for the 
purposes of this study. For at least the past five years, community colleges in each state 
have been supported by statewide agencies or associations in implementing student 
success reforms based on the guided pathways reform framework. Two states (B and C) 
have had program-level transfer agreements in place for 10 years, while State A more 
recently began developing program-level agreements that build on a long-standing set of 
common general education courses, introductory program courses, and course 
numbering. As a result of these statewide efforts, stakeholders from community colleges 
and four-year institutions in all three states have established STEM transfer pathways 
designed to provide community college STEM transfer students with entry into specific 
STEM majors with junior standing at the state’s public (and in some cases private) four- 
year institutions. Of States A, B, and C, all three have established pathways for biology, 
chemistry, physics, geology, engineering, and mathematics majors; two have established 
pathways for computer sciences and information technology majors; and one has an 
established pathway for agricultural sciences majors. Each transfer pathway includes a set 
of specific lower-division courses that are commonly agreed by public universities and 
community colleges in each state to fulfill requirements for a bachelor’s degree in the 


given major. Each transfer pathway also provides guidelines for other general education 
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or elective courses that students should take during their first two years at community 
college in order to enter as a junior in the major at a public university. Typically, 
pathways are initially developed and periodically updated by discipline-specific faculty 
work groups including representatives from both two- and four-year institutions, 
convened by state agencies or other statewide organizations. 

For example, in one state, the chemistry major transfer pathway includes the 
courses Calculus I and II, Calculus-Based Physics I and II with lab, General Chemistry I 
and II with lab, and Organic Chemistry I and II with lab. Outside of these specific STEM 
courses, the pathway also provides clear guidelines for selecting additional lower- 
division courses required to complete a bachelor’s degree at any public university (e.g., 
English Composition I and II, an arts and humanities requirement, a social and behavioral 
sciences requirement). Transfer pathways may mention, but do not list as requirements, 
certain institution-specific prerequisites. For example, students who did not take 
trigonometry or precalculus in high school may need to complete one of these courses 
prior to enrolling in Calculus I. Similarly, students who did not complete a full year of 
physics in high school may need to take Algebra-Based Physics before enrolling in 
Calculus-Based Physics I. 

In all three states, the statewide STEM transfer pathways are used as advising 
guides for transfer-intending community college students and provide clear roadmaps to 
guaranteed entry into STEM mapjors after transferring to one of the state’s public 
universities. In the current study, we focus on the STEM courses listed on these transfer 
pathways as our primary indicators of interest, guided by the hypothesis that increased 
enrollment in these specific courses early in a student’s community college career should 


predict longer-term outcomes like STEM transfer and bachelor’s degree completion. 


2.2 Gathering Data 

We analyze state-collected administrative records of students who started for the 
first time in community college with an intent to transfer and complete a bachelor’s 
degree across three state higher education systems, including 70 community colleges and 
268,197 students. In States B and C, administrative records are available only for the 
community college sector; however, State A records also include the public four-year 


sector. To assess whether our findings on STEM momentum indicators may also be 
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applicable to the outcomes of similar students beginning in the four-year sector, including 
those students who declare a baccalaureate STEM major, our analysis also includes 
56,577 students who began at 26 nonselective regional four-year institutions in State A.! 
Each state’s administrative student record databases include a set of student 
demographic and other characteristics, term-by-term enrollment records, detailed 
transcript records, and degree and credential completion records, along with linked 
National Student Clearinghouse records on enrollments and degree completions 
nationally. These administrative data include all degree-seeking students who entered a 
public state college as a first-time student between fall 2010 and summer 2014 and track 


each student’s record for six years after starting college. 


2.3 Estimating STEM Transfer Outcomes 

Our primary outcome of interest is whether a student earns a bachelor's degree in 
a STEM field within six years of their first college entry. The STEM designation of a 
particular field is based on Wang’s (2016) classification of two-digit CIP codes, with the 
most common bachelor’s degree STEM CIP codes being 01 (agricultural, animal, plant, 
veterinary science, and related fields), 03 (natural resources and conservation), 11 
(computer and information sciences and support services), 14 (engineering), 26 
(biological and medical sciences), 27 (mathematics and statistics), and 40 (physical 
sciences). In addition to STEM bachelor’s degree completion, we also examine non- 
STEM bachelor’s degree completion as a useful standard of comparison to STEM 
completion. 

Because completion of a bachelor’s degree requires many years to measure, and 
relatively few community college students complete a bachelor’s degree within six years, 
in State A we also capture two intermediate outcomes that may occur along a successful 
path toa STEM bachelor’s degree: (1) transfer to a four-year institution and enrollment in 
a STEM major within three years of initial community college entry, and (2) progress in 
STEM after transfer and enrollment in a STEM major. For this second intermediate 


outcome, we assess whether community college entrants who transfer into a STEM major 


1 In an ancillary analysis presented in section 3.4 we also include 127,017 other students who 
entered a middle-selective four-year institution in State A. 
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at a four-year institution (intermediate outcome 1) complete nine or more college credits 


in STEM fields during their first year post-transfer. 


2.4 Classifying Early Coursetaking Variables 


Drawing on each state’s STEM transfer pathways, we build on Wang’s (2016) 


and Chan and Wang’s (2020) classification of early community college STEM 


coursetaking by detailing 10 different types of early community college STEM courses, 


as summarized in Table 1. 


Table 1 


Community College STEM Course Typology 


Math course type Definition Examples 
Calculus Any calculus course Calculus I/II 
Precalculus 


Calculus foundation 


Statistics 


Other college-level 


Any college-level precalculus, trigonometry, geometry, 
or algebra course 


Any college-level statistics course 


Any other type of college-level math course outside of 


Trigonometry/Geometry 
College Algebra 


Introduction to Statistics 


Differential Equations 


math the categories above Accounting 
Developmental math Any developmental or remedial math course Pre-Algebra 
STE course type Definition Examples 
Specified in statewide transfer agreements as STE ; 
: ; . : Chemistry I/II 
course that applies to university STEM major program : 
STE pathway Sk eaten » Biology I/II 
and enables students to enter university “major-ready Physics 


STE foundation 


Other STE, likely 
transferable 


Other STE, likely 
terminal 


Any STE 


in STEM 


College-specific prerequisite course for the STE 
pathway courses 


Likely transferable course based on two-digit CIP codes 
from Wang (2016); not included in pathway or 
foundation categories 


Likely terminal (career and technical education) course 
based on two-digit CIP codes from Wang (2016); not 
included in pathway or foundation categories 


Broad STE definition that includes any of the courses 
above and all courses with STE CIP codes based on 
Wang’s (2016) classification but excluding math courses 
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Intro to Chemistry 
Intro to Biology 
Intro to Physics 


Intro to Computers 
Nutrition 
Astronomy 


Drafting 
Information Technology 


All of the above 


Math courses are commonly required for many majors both within and outside of 
STEM. Within STEM, transfer-level science courses commonly build on skills developed 
in various math courses; as a result, math and science courses are often taken 
concurrently or one after another in subsequent semesters. This dual approach to math 
and science course completion could obfuscate any program-specific findings; if we were 
to measure STEM coursetaking in general, its predictive power could be due merely to 
generic math momentum. Thus, our typology of community college STEM coursework 
organizes courses into separate categories for math and non-math science, technology, 
and engineering (STE) subjects. We further organize courses according to whether they 
specifically appear on the state transfer pathway (“pathway”) or serve as a prerequisite to 
such courses (‘foundation’). As a result, our typology is built around four key types of 
STEM courses: pathway math (calculus), pathway non-math (STE pathway), 
foundational math (calculus foundation), and foundational non-math (STE foundation). 
The following subsections describe our approach to identifying these types of courses, as 
well as how we categorize the other types of STEM courses that do not fit into pathway 
or foundation categories. 

Math courses. To classify courses into a math subject area, we first used course 
CIP codes to derive a preliminary list of all math courses and then conducted a manual 
inspection of keywords in course titles to ensure courses are accurately classified by 
domain. Based on our review, we categorize each math course into one of nine topics: 
calculus, precalculus, trigonometry, geometry, college-level algebra, statistics, other 
college-level math, or developmental math. Of all math courses we analyze, calculus is 
the only topic that commonly appears within each state’s STEM transfer pathways. 

To identify the college-level courses that commonly serve as prerequisites to 
calculus, we examined a broad selection of community college course catalogs within 
each state. Although specific prerequisites vary across institutions, within all three states, 
college algebra, trigonometry, geometry, and/or precalculus are common prerequisites; 
thus, all instances of courses in these subjects are classified as “calculus foundation.” 

To avoid conflating calculus foundation courses with non-college-level courses 
such as Pre-Algebra, we maintain a separate classification for developmental math 


courses. Finally, we also maintain separate classifications for statistics courses (which 


14 


commonly appear on non-STEM transfer pathways in each state) and for other college- 
level math courses. Definitionally, other college-level math includes any college-level 
course that has a math CIP code but does not fit into any of the prior categories. Such 
courses may be transferable or not; in general, they are irrelevant to STEM transfer 
pathways (such as the courses Technical Mathematics or Quantitative Reasoning) or are 
higher-level courses rarely taken by community college students (such as Differential 
Equations). Given that the various subtypes within the other college-level math category 
are relatively uncommon in our dataset, we believe that grouping them together is an 
appropriate choice for this study. 

STE pathway courses. For each state, we identified non-math STE courses that 
appear on at least one STEM transfer pathway. For example, within one state, Calculus- 
Based Physics I appears on the physics, chemistry, and math transfer pathways and is 
thus classified as a STE pathway course for that state. Although the lists of STE pathway 
courses are similar across the three states, they are not identical. For example, Historical 
Geology appears on a STEM transfer pathway and is thus classified as a STE pathway 
course in two states; however, it or a similar course does not qualify as a STE pathway 
course in the remaining state, so it is not included on the list for that state. 

STE foundation courses. Similar to how calculus foundation courses function in 
relation to calculus courses, STE foundation courses are prerequisites for STE pathway 
courses. Because prerequisites vary by state and institutional context, the number and 
type of these courses, though generally similar across contexts, vary as well. For 
example, some community colleges have multiple prerequisites for a given STE pathway 
course, while others offer only one prerequisite or none at all. In each state, we examined 
course catalogs and conducted conversations with state and college informants to identify 
relevant prerequisites offered by any community college in the state. For example, in one 
state, Algebra-Based Physics is considered a prerequisite for Calculus-Based Physics at 
many but not all of the state’s community colleges; in that state, all instances of Algebra- 
Based Physics are classified as STE foundation courses. Often these college-level courses 
replicate coursework that is commonly offered in high schools but is not always required 


for high school graduation. 


ie) 


Other STE courses. While the guided pathways framework suggests that courses 
appearing on STEM transfer pathways are likely to be the most important markers of 
successful transfer, it may also be important to capture other types of STEM 
coursetaking. Thus, we used Wang’s (2016) identification of likely transferable and 
likely terminal STEM CIP codes to categorize the remaining non-math STE courses 
offered in all three states. To ensure we observe any overall relationship between STEM 
transfer outcomes and early STE coursework—defined in the broadest terms—we created 
a tenth and final category of STEM coursetaking, any STE, which includes any courses in 


the STE pathway, STE foundation, and other STE categories. 


2.4 Identifying a Simple Set of Predictive Metrics 

To explore our first research question—Can a simple set of STEM momentum 
metrics predict students’ long-term transfer outcomes at a similar or superior level as 
widely used general early momentum metrics?—-we first run a series of logistic 
regression models. As described in Table 2, we group a wide array of potentially useful 
independent variables into different combinations or “blocks” of predictors. Grouping 
these variables into different blocks allows us to compare how well different sets of 
predictors explain variation in each of the longer-term STEM transfer outcomes. In other 
words, we can compare whether a limited set of STEM momentum metrics explains 
about the same amount of variation in students’ longer-term outcomes as a much more 
detailed set of STEM momentum metrics. We take this first step in our analysis in order 
to compare different sets of STEM momentum metrics to general academic momentum 
metrics and to assess how much additional explanatory power is gained by adding more 
predictors to the model. The following subsections describe the predictors included in 


each block. 
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Table 2 
Block Design for Logistic Regression 


Block Description of variables included 


Year 1 average GPA > 3.0 

Completed 6+ college-level credits in term 1 

Completed 12+ college-level credits in term 1 
General academic momentum? Year 1 persistence (enrolled in multiple terms) 

Completed college-level English credits in year 1 

Completed 12+ college-level credits in year 1 

Completed 24+ college-level credits in year 1 


Year 1 college-level math GPA > 3.0 

Completed calculus credits in year 1 

Completed calculus foundation credits in year 1 
Completed statistics credits in year 1 

Completed other college-level math credits in year 1 
Completed developmental math credits in year 1 


Math momentum 


Year 1 STE GPA > 3.0 

Completed any STE credits in year 1 

Completed STE foundation credits in year 1 

Completed STE pathway credits in year 1 

Completed other STE credits that are likely transferable in year 1 
Completed other STE credits that are likely terminal in year 1 


STE momentum 


Expanded STEM momentum (STE 


All variables included in both math momentum and STE momentum blocks 
and math momentum) 


Completed calculus credits in year 1 

Completed calculus foundation credits in year 1 
Completed STE foundation credits in year 1 
Completed STE pathway credits in year 1 


Parsimonious STEM momentum 


Student demographics Age, gender, income, race/ethnicity 


4Credits reported in semester hours were converted and analyzed in equivalent quarter hours where relevant. 


General academic momentum, math momentum, and STE momentum 
blocks. The general academic momentum block includes indicators of general academic 
momentum from Belfield et al. (2019), and the math momentum and STE momentum 
blocks focus on potential math-specific or STE-specific indicators. The general academic 
momentum block includes indicators such as whether students earn at least 6 or 12 credits 
in their first term, earn at least 12 or 24 credits in their first year, complete college-level 
English credits in their first year, persist from their first term to second term, and 
maintain a 3.0 GPA or higher in their first year (Belfield et al., 2019). The math 
momentum block includes indicators such as whether students complete a math course 


categorized as calculus, precalculus, trigonometry, or geometry, college-level algebra, 
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developmental math, or statistics, and whether students earn a B or higher GPA in math 
coursework. The STE momentum block includes indicators such as whether students 
complete STE pathway courses, STE foundation courses, or other likely transferable or 
likely terminal STE courses classified with a CIP code taxonomy from Wang (2016), 
excluding math coursework. In the STE momentum block we also include indicators such 
as whether students earn a B or higher GPA in their STE coursework. 

Expanded and parsimonious STEM momentum blocks. Together, the math 
momentum, STE momentum, and general academic momentum blocks contain the full 
array of potentially useful momentum indicators. In order to understand whether a full 
array is necessary or a more parsimonious array may be as useful, we design two 
additional blocks. First, the expanded STEM momentum block includes all 12 variables 
from the math and STE momentum blocks. Second, the parsimonious STEM momentum 
block includes only four key STEM momentum variables: completion of calculus, 
calculus foundation, STE pathway, or STE foundation courses. Based on our review of 
statewide transfer pathways and feedback from state agencies and college faculty and 
staff, we hypothesize that these four indicators best capture early STEM momentum. In 
particular, completion of calculus and STE pathway coursework enables students to enter 
a STEM baccalaureate major after transfer, and thus we expect completion of these two 
types of early STEM coursework to be most strongly related to STEM transfer outcomes 
relative to our other indicators. However, students may not have progressed to these more 
advanced STEM pathway courses in their first year of college; thus, we also include in 
the parsimonious STEM momentum block prerequisites to calculus and STE pathway 
courses (i.e., calculus foundation and STE foundation courses). 

Demographic indicators. Available student demographic data varies by state but 
in each state includes at least race/ethnicity, gender, age, and a measure of student/family 
income (e.g., eligibility for need-based aid). 

To explore how well the parsimonious STEM momentum block explains STEM 
transfer outcomes relative to other blocks with more predictor variables, we construct a 
series of models predicting six-year bachelor’s degree attainment in STEM using 


different combinations of blocks. The model equations are specified as follows: 
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STEM BA in 6yr, = a; + BCSTEM Block 3;) + y(Academic Momentum;) + 
6X; + hi + &; (1) 


STEM BA in 6yr,; = a; + B(STEM Block 4;) + y(Academic Momentum;) + 
5X; +h; + &; (2) 

In these equations, 6 represents a vector of STEM credit momentum binary 
indicators for student i (STEM Block 3 represents all STEM momentum measures while 
STEM Block 4 represents the parsimonious set of four STEM momentum measures), 

y represents a vector of early academic momentum binary indicators for student i, 6 
includes controls for student demographics, @ represents an institution-by-year fixed 
effect, and € is an error term clustered at the institution-year level. In State A, parallel 
models are also conducted with the binary outcomes of transfer into a STEM major and 
STEM progression post-transfer. 

In order to compare the predictive power of different combinations of blocks, we 
examine pseudo-R? as well as visuals of the area under the ROC (Receiving Operating 
Characteristic) curves (AUC). AUC values range from 0 to 1, with values of .5 and below 
indicating limited to no predictive power and values of .8 and above indicating acceptable 


predictive ability (Bowers & Zhou, 2019). 


2.5 Examining the Preferred Set of Metrics 

Our second research question asks whether a simple set of STEM momentum 
metrics are reliable (or consistent in their predictive power) across a wide variety of 
student groups, including those that are historically underrepresented in STEM. Given the 
broad array of state, institutional, and program contexts under study, it may be that, 
although preferred metrics are associated with subsequent STEM transfer outcomes 
overall, there are particular subgroups of students to whom this overall pattern does not 
apply. If we are able to demonstrate that the parsimonious STEM momentum block 
provides a good balance between simplicity and predictive power, we will further 
examine the reliability of these measures in two ways. First, in order to ensure that our 
findings are not merely artifacts of variation in STEM program quality or performance 
across institutions or cohorts, we will expand Equation 2 by adding institution-by-year 
fixed effects to control for cross-institution and cross-year variation. Second, we will take 


two different approaches (a “conditional” approach and an “interaction” approach) to 
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examine reliability across student gender and race/ethnicity. In the conditional approach, 
we will model Equation 2 conditional on race/ethnicity or gender to allow comparisons 
across student subgroups in terms of the predictive power of our parsimonious STEM 
momentum block. In the interaction approach, we will expand Equation 2 by adding a 
vector of interaction terms for each of the parsimonious STEM momentum block 
predictors, in order to test for disproportionate effects for female students and for students 
from historically underrepresented racial/ethnic groups (Black, Hispanic, Native 


American, and Pacific Islander). 


2.6 Exploring Practical Insights 

Our third research question deals with the practical meaning of these metrics. 
That is, to what extent do these metrics reflect students’ intent to study STEM, students’ 
success within STEM, and institution-specific efforts to support STEM pathways? 

To explore this question, we first must understand whether these metrics capture 
students’ STEM intent, STEM momentum, or both. Although most community colleges 
track their students’ declared majors, transfer-intending students are typically enrolled in 
generic majors (e.g., “general transfer’’), leaving colleges uncertain about whether a 
student intends to transfer in STEM. Thus, any predictive power of early community 
college STEM momentum indicators could be derived primarily from their signaling 
effect—the indicators’ ability to identify STEM transfer-aspiring students. To explore 
this issue, we take two different approaches in an attempt to disentangle STEM transfer 
intent and momentum. First, we examine the predictive power of our preferred STEM 
momentum indicators if we require only that students attempt, rather than complete, these 
courses in their first year. Earlier work on general academic momentum suggests that 
even attempting more credits is associated with stronger college outcomes (Attewell et 
al., 2012). Furthermore, attempting a course, such as a calculus or STE pathway course, 
may represent an equivalent signal of student intent to transfer into STEM as completing 
that same course. Second, we draw on samples of entrants at nonselective public four- 
year colleges in State A, for whom we can observe STEM bachelor’s degree major intent. 
We compare the predictive power of the preferred STEM momentum metrics versus the 
declaration of a STEM major among this sample of four-year entrants. We further 


extrapolate this comparison to the community college population in the same state by 
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testing the reliability of findings among a sample of community college students who are 
matched to peers at nonselective four-year colleges who declared a STEM major. 
Finally, we use our preferred STEM momentum metrics to understand the extent 
to which colleges are helping students gain STEM transfer momentum during the period 
under study. To do so, we provide descriptive information regarding variation across 
community colleges, as well as disparities by gender and race/ethnicity, to get a sense of 
whether colleges are helping students complete key STEM courses and gain STEM 


transfer momentum. 


2.7 Identifying Limitations 

Our analysis makes use of information recorded in statewide administrative data 
systems, which do not provide data on precollege factors that influence academic 
outcomes, such as high school performance measures and high school attended. While 
our logistic regression analysis and ensuing robustness checks are high powered, the lack 
of an experimental or quasi-experimental design prevents us from making causal claims. 
Our analysis is correlational: We ask which readily available early indicators can predict 
students’ outcomes, with the understanding that these proximal indicators and the 
ultimate outcomes of interest may be mutually influenced by other student-level and 
institutional-level factors. 

Another limitation of our study is a lack of information regarding community 
college student majors. As noted earlier, community college students who intend to 
transfer are typically enrolled in generic majors such as “general transfer.” Our 
propensity score matching analysis shows that early STEM momentum metrics capture 
more than just STEM intent; however, we could likely improve upon our preferred 
model’s predictive power if we had more robust data on community college students’ 
major intent. 

Finally, although we work carefully to connect courses to statewide pathways, we 
may not always accurately categorize STEM courses. It is particularly challenging to 
identify course prerequisites; these courses are not designated on statewide pathways, 
which contributes to an astonishing degree of variation in STEM prerequisite structures 


and course titles across the 70 community colleges and 26 non-selective four-year 
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institutions included in this study. As a result, some courses that should belong in the 


STE foundation category may be erroneously classified as other STE, likely transferable. 


3. Results 


3.1 Sample 

Tables 3 and 4 present descriptive results on the demographic characteristics of 
our student sample in each state as well as the independent variables included in the 
general academic and STEM momentum blocks. In students’ first year at community 
college (CC), between 36% and 43% of students complete any non-math STE college- 
level coursework, while between 19% and 34% of students complete any college-level 
math coursework. The most commonly completed STEM courses are those in the 
category of other STE, likely transferable, while the least commonly completed STEM 
courses are those in the category of other STE, likely terminal. The proportion of students 
completing non-math STE foundation or pathway courses varies across states but remains 
below 10% in each state included in our sample. Only 7% of students in State A, where 
four-year institutional data are available, transfer into a STEM major within three years 
of starting at community college. In State A, among students who transfer into STEM, 
63% accumulate nine or more STEM credits in their first year post-transfer. Overall, 
STEM bachelor’s completion rates are low for community college starters, with no state 
in our sample exceeding 4% among all entering transfer-intending students (although 


between 8% and 18% of students complete a bachelor’s degree in a non-STEM field). 
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Table 3 
Sample Characteristics 


State A, CC State A, NSFY State B, CC State C, CC 
Student characteristics entrants entrants entrants entrants 
(N = 92,679) (N= 56,577) (N = 50,890) (N = 124,628) 

Gender 

Male 48% 45% 44% 52% 

Female 52% 55% 56% 48% 
Race/ethnicity 

Asian 2% 1% 2% 12% 

Black 22% 12% 20% 6% 

Native/American Indian 1% <1% <1% 1% 

Hispanic 4% 2% 4% 3% 

Pacific Islander <1% <1% <1% < 1% 

White 64% 77% 70% 55% 

Other/Unknown 7% 7% 4% 15% 
Age upon entry 

20 and under 64% 73% 68% 60% 

21-24 17% 11% 12% 16% 

25+ 19% 17% 19% 24% 
Entering cohort 

2010-11 AY 27% 27% 26% 27% 

2011-12 AY 26% 26% 26% 26% 

2012-13 AY 25% 24% 25% 23% 

2013-14 AY 22% 23% 23% 24% 


23 


Table 4 
Outcomes and Momentum Indicators 


State A, State A, State B, State C, 
cc NSFY cc cc 
(N = 92,679) (N = 56,577) (N=50,890) (N=124,628) 
Outcomes 
STEM bachelor’s completion 3% 3% 1% 4% 
Non-STEM bachelor’s completion 15% 18% 8% 15% 
Transfer into STEM major 7% 
Persistence in STEM in first year post-transfer 
4 63% 
(among those who transfer into STEM) 
General academic momentum 
Year 1 average GPA > 3.0 36% 31% 26% 38% 
Completed 6+ CL credits in term 1 48% 62% 49% 62% 
Completed 12+ CL credits in term 1 17% 26% 20% 25% 
Year 1 persistence (enrolled in multiple terms) 75% 78% 82% 66% 
Completed CL English in year 1 47% 56% 45% 44% 
Completed 12+ CL credits in year 1 48% 59% 41% 53% 
Completed 24+ CL credits in year 1 23% 33% 16% 26% 
Math momentum in year 1 
Year 1 CL math GPA > 3.0 44% 39% 42% 46% 
Completed any CL math credits 24% 34% 19% 23% 
Completed calculus credits 4% 5% 1% 6% 
Completed any calculus foundation credits 16% 25% 10% 9% 
Completed calculus foundation credits: precalculus 2% 3% 3% 9% 
Completed calculus foundation credits: 2% 5% <1% <1% 
trigonometry or geometry 
Completed calculus foundation credits: CL algebra 12% 17% 6% 2% 
Completed statistics credits 4% 4% 8% 5% 
Completed other CL math credits 6% 8% 7% 5% 
Completed developmental math credits 34% 30% 15% 37% 
STE momentum in year 1 
Year 1 STE GPA > 3.0 45% 38% 45% 46% 
Completed any STE credits 36% 43% 26% 29% 
Completed any STE foundation credits 5% 4% 2% 6% 
Completed any STE pathway credits 6% 8% 6% 3% 
Completed other STE credits, likely transferable 33% 42% 25% 26% 
Completed other STE credits, likely terminal 4% 2% 1% 2% 
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3.2 Predictive Power of STEM Momentum Metrics 

Our first research question asks whether a simple set of STEM momentum 
metrics can predict students’ long-term outcomes at a similar or superior level as widely 
used general early momentum metrics. In order to explore that question, Tables 5 and 5a 
provide the pseudo R? for each of our logistic regression models, with Table 5 focusing 
on six-year STEM bachelor’s degree completion and Table 5a focusing on the 
intermediate outcomes in State A. As expected, explanatory power increases substantially 
as we move from including only a single block to including multiple blocks in the same 
model. However, the parsimonious STEM momentum block, comprising only four math 
and science coursetaking momentum indicators, explains an equivalent or slightly larger 
amount of variation when compared to general academic momentum, math momentum, 


or STE momentum blocks. 


Table 5 
Logistic Regression Model Summary, STEM Bachelor’s Degree Completion 


Pseudo R? 
(observations) 


wWorable bineks State A, State A, State B, State B, 
CC NSFY CC CC 
NiSeirmiciwentin: 187 .193 .203 147 
(91,894) (56,137) (50,607) (123,668) 
ere enonienturn .203 .213 175 119 
(91,929) (56,150) (50,607) (123,668) 
P : .216 .216 215 151 
Parsimonious STEM momentum (91,929) (56,150) (50,607) (123,668) 
: .257 .269 .247 .182 
Expanded STEM momentum (blocks 1 and 2 combined) (91,894) (56,137) (50,607) (123,668) 
General academic momentum ae ab3 88 igs 
(91,890) (56,138) (50,607) (123,668) 
: .273 .286 .266 .207 
General academic and STEM momentum (91,882) (56,132) (50,607) (123,668) 
General academic and STEM momentum, student .295 .299 .283 .223 
demographics (91,835) (54,506) (50,607) (123,668) 
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Table 5a 
State A, Community College Entrants: 
Logistic Regression Model Summary, Intermediate STEM Transfer Outcomes 


Pseudo R2 
(observations) 


STEM progression 


Variable blocks Transfer into STEM major 
post-transfer 
163 .246 
Math momentum (92,642) (92,481) 
181 .263 
STE momentum (92,679) (92,517) 
: , .209 301 
Parsimonious STEM momentum (92,679) (92,517) 
: 235 343 
Expanded STEM momentum (blocks 1 and 2 combined) (92,642) (92,481) 
General academic momentum oz oe 
(92,638) (92,477) 
: .242 359 
General academic and STEM momentum (92,630) (92,469) 
General academic and STEM momentum, student .292 392 
demographics (92,583) (92,422) 


Figure | presents AUC values for each model in Table 5. Across states and 
models, AUC values range from approximately .8 to .9 and are highest for the full model 
that includes general academic, math, and STE momentum and demographic variable 
blocks. While there is variation in AUC values across states and models, their relative 
stability and high level suggest that all the models—including the model containing only 
the parsimonious STEM momentum block—are reliably strong predictors of STEM 
bachelor’s degree completion for community college entrants in all states, as well as for 
four-year entrants in State A. Similar AUC results (not shown) are observed for the 


models in Table Sa. 
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Figure 1 
AUC Values for Each Model Block by State (STEM Bachelor’s Degree Completion) 
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The R? and AUC results suggest that the four indicators included in the 
parsimonious STEM momentum block are jointly useful in explaining STEM bachelor’s 
degree completion. To explore the practical descriptive power of these four predictors 
(and how they compare to the other potential predictors), Table 6 shows the descriptive 
relationship between early STEM coursetaking and STEM bachelor’s degree completion. 
While only between 1% and 4% of students in our full sample complete a STEM 
bachelor’s degree within six years of entering community college, those who complete a 
STE pathway or calculus course in their first year have substantially higher rates of 
STEM bachelor’s completion, particularly in States A and C. For example, among 
students in State C who complete a STE pathway course in their first year at community 
college, 25% complete a STEM bachelor’s degree within six years. We observe 
heterogeneity across states in the relationship between degree completion and calculus or 
STE pathway course completion in students’ first year, with State B having substantially 
lower rates of STEM bachelor’s completion, both conditional on meeting these metrics 
and overall. However, calculus or STE pathway course completion is still among the 
strongest descriptive predictors of STEM bachelor’s completion in State B. 

While completing a calculus or STE pathway course is a clear predictor of longer- 
term STEM transfer success, this is less clear for calculus foundation and STE foundation 
courses. Descriptively, in each state, completion of a calculus foundation course seems to 
be a more useful indicator of STEM bachelor’s completion than any of the general 
academic momentum metrics alone, albeit less useful than completion of calculus or STE 
pathway credits. However, in each state, completing a specific calculus foundation course 
may be an equivalent or better predictor than completing any calculus foundation course. 
For example, in States A and B, completing precalculus credits may be a more useful 
descriptive indicator, while in State C, completing trigonometry or geometry credits may 
be more useful. Completion of a STE foundation course is a relatively weak descriptive 
predictor compared to the other three parsimonious indicators; however, it performs 
reasonably well in comparison to general academic momentum indicators. 

Further corroborating findings from the regression analyses, descriptive results 
presented in Table 6 suggest that early STEM indicators—particularly calculus and STE 


pathway credit completion and to a lesser degree calculus and STE foundation credit 
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completion—have a stronger relationship with STEM bachelor’s degree completion than 
do general measures of academic momentum. Taken together, these findings suggest that 
the parsimonious set of four STEM momentum metrics are generally suitable as leading 
indicators of longer-term STEM outcomes across the state samples.” In subsequent 
analyses, we examine the reliability of this parsimonious set of four STEM indicators 


across institutions, entry cohorts, and student demographic subgroups. 


Table 6 
STEM Bachelor’s Degree Completion Rates Conditional on First-Year Coursetaking Metrics 


Six-year STEM bachelor’s completion rate 


State A, State B, State C, 
Characteristic cc state Cee cc cc 
(N = 56,577) 
(N = 92,679) (N = 50,890) (N = 124,628) 

All students 3% 3% 1% 4% 
General academic 
momentum 

Year 1 average : = ‘ ‘ 

GPAS3.0 6% 7% 2% 7% 

Completed 6+ 

CL credits in 5% 5% 1% 4% 

term 1 

Completed 12+ 

CL credits in 10% 8% 3% 2% 

term 1 

Year 1 

persistence A% A% 1% A% 

(enrolled in 

multiple terms) 

Completed CL 

English credits 4% 4% 1% 5% 

in year 1 

Completed 12+ 

CL credits in 6% 5% 2% 5% 

year 1 

Completed 24+ 

CL credits in 9% 8% 3% 6% 


year 1 


STEM momentum in 
year 1 


? As acheck on this interpretation, we also examine whether these four STEM momentum indicators are 
generally predictive of earning any bachelor’s degree rather than a STEM bachelor’s degree. Across all 
samples, we find that the four STEM momentum metrics are generally weak predictors of completion of 
non-STEM bachelor’s degrees, with most individual coefficients being not significant. That is, the STEM 
momentum indicators appear to be precise in capturing early STEM program momentum; they predict 
subsequent transfer success in STEM but not in non-STEM. 
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Completed 


: 26% 23% 14% 23% 
calculus credits 


Completed any 
calculus 
foundation 
credits 


16% 12% 4% 11% 


Completed 

calculus 

foundation 17% 15% 9% 11% 
credits: 

precalculus 


Completed 
calculus 
foundation 
credits: 
trigonometry 
or geometry 


14% 10% 4% 15% 


Completed 

calculus 

foundation 6% 5% 2% 10% 
credits: CL 

algebra 


Completed any 
STE foundation 7% 9% 3% 9% 
credits 


Completed any 
STE pathway 24% 18% 6% 25% 
credits 


Note. This table reports rates of STEM bachelor’s completion within six years of beginning in college, conditional on 
students’ first-year course completions. Credit thresholds are reported as semester-system equivalents. 


3.3 Reliability of STEM Momentum Indicators 

Our second research question asks whether our preferred metrics are reliably 
useful across a wide variety of student groups, including those that are historically 
underrepresented in STEM. First, we examine reliability across institutions and entry 
cohorts; next, we examine reliability across gender and race/ethnicity. 

Reliability across institutions and entry cohorts. Table 7 presents results from a 
series of logistic regression models that examine how well each of the four parsimonious 
STEM momentum indicators predicts subsequent STEM bachelor’s completion net of 
other general academic momentum indicators, student demographics, and differences 
observed by institution and entry year. In particular, we focus on Model 3, which 
includes institution-by-year fixed effects alongside other controls to test the reliability of 


the four STEM momentum indicators across unobservable differences based on the 
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college or cohort year in which students begin their journey toward a STEM bachelor’s 
degree. 

While coefficients for each of the four indicators attenuate from Model | to 
Model 3 within each state, they appear to be more stable for STE pathway and calculus 
course completion. For instance, in State A, the coefficients for STE pathway course 
completion attenuate only slightly across models from roughly .05 to .04; for State A 
four-year colleges, the coefficient for calculus course completion also drops only slightly 
across models from about .04 to .03. 

After controlling for demographic, institutional, and cohort fixed effects in Model 
3, in States A and C the coefficient estimates for STE pathway and calculus course 
completion range from .02 to .04 and are consistently statistically significant at p < .001. 
These results suggest that early completion of a STE pathway or calculus course in these 
states is associated with a two- to four-percentage point increase in the likelihood of 
attaining a STEM bachelor’s degree within six years of entry. While these single digit 
percentage point increases may seem small, they represent a meaningful increase given 
that only between 1% and 4% of students in the sample transfer and complete STEM 
bachelor’s degrees overall. In State B, coefficient estimates for STE pathway and 
calculus course completion appear slightly less strong but remain positive and 
statistically significant. Thus, in general, STE pathway or calculus course completion 
appears to be a useful predictor of STEM bachelor’s degree attainment regardless of 
state, institutional, or cohort contexts. 

Consistent with the descriptive results presented in Table 6, coefficients in Table 
7 for STE and calculus foundation course completion are weaker than for STE pathway 
and calculus course completion. In some models and states, the foundation indicator 
coefficients are weakly positive and significant, while in others, they are estimated as 
near zero. Thus, while STE or calculus foundation course completion has some general 


predictive value, it may be less useful within certain state and institutional contexts. 
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Table 7 
Regression Estimates From Predictive Models of First-Year Coursetaking on Six-Year STEM Bachelor’s Degree Completion 


State A, CC State A, NSFY State B, CC State C, CC 


Model 1 Model 2 Model 3 Model 1 Model 2 Model 3 Model 1 Model 2 Model 3 Model 1 Model 2 Model 3 


Completed STE 
pathway credits 0.051*** 0.040***  0.039*** | 0.046***  0.037***  0.036*** | 0.013*** 0.010***  0.010*** | 0.043***  0.040***  0.041*** 
in Y1 


(0.001) (0.001) (0.012) (0.002) (0.002) (0.002) (0.001) (0.001) (0.001) (0.001) (0.001) (0.002) 
Completed STE 
foundation 0.007*** 0.001 0.000 0.012***  0.009***  0.010*** | 0.006***  0.004***  0.004*** | 0.008***  0.008***  0.008*** 
credits in Y1 

(0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) 
Completed 


calculus credits 0.046*** 0.029***  0.024*** | 0.045***  0.033***  0.030*** | 0.013*** 0.009***  0.008*** | 0.059***  0.038***  0.037*** 
in Y1 


(0.001) (0.001) (0.001) (0.002) (0.002) (0.002) (0.001) (0.001) (0.001) (0.001) (0.001) (0.002) 
Completed 
ee 0.018*** 0.003* 0.002 0.027*** 0,019*** 0.018*** | 0.010*** 0.005*** 0.006*** | 0.027*** 0.015*** 0.016*** 
credits in Y1 

(0.001) (0.001) (0.001) (0.002) (0.002) (0.002) (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) 
Observations 92,679 90,567 90,411 56,577 56,565 54,510 50,890 50,890 50,607 124,628 124,628 123,668 
Academic x x x x 
controls 
Demographic x x x x x x x x 
controls 
Institution & 
cohort fixed xX xX xX xX xX X X X 
effects 


Note. Standard errors in parentheses. 


#8 < 001. **p <.01. *p<.05. 
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Reliability across race/ethnicity and gender. In Table 7, Model 2 controls for 
demographic characteristics such as race/ethnicity and gender, demonstrating that the 
four parsimonious STEM indicators are still statistically significant predictors of STEM 
bachelor’s degree completion after controlling for these characteristics. To explore 
potential heterogeneity in the predictive power of STEM momentum metrics across 
race/ethnicity and gender, Figure 2 presents the pseudo R? values derived from Equation 
2 conditioned on different student racial/ethnic and gender groups by state. In each state, 
the R? values for historically underrepresented groups are similar to or better than the R? 
values for White or male students, suggesting that the four parsimonious STEM 


indicators are indeed predictive for historically underrepresented students. 


Figure 2 
R? Value of Parsimonious STEM Momentum Block by Race/Ethnicity and Gender 
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As an additional check, we test for disproportionate effects for historically 
underrepresented students by expanding Equation 2 with a vector of interaction terms by 
gender (“FEM”) and historically underrepresented students of color (“HU”), including 
Black, Hispanic, Native American, or Pacific Islander students (see Appendix Table A1). 
In general, the results suggest that the four STEM momentum metrics are reliable across 
race/ethnicity and gender. While there are a few statistically significant interactions 
across the four indicators, they do not form a consistent pattern. A few interactions in 
State A imply that the STEM momentum metrics may have slightly more predictive 


power for women. In only one case does an interaction effect nullify or reverse the 
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direction of the main effect for a parsimonious indicator: In State C, after taking into 
account a significant and negative interaction effect, the relationship between STE 
foundation coursework and STEM bachelor’s completion is erased and even dips into 
negative territory for historically underrepresented students of color. To further 
understand this relationship, more exploration closer to practice is needed, including an 
investigation of inequities in student referral to STE foundation courses and availability 
of STE pathway courses. 

To further describe the relationship between key early STEM course completion 
and longer-term STEM bachelor’s degree completion, Table 8 reports STEM bachelor’s 
completion rates for students who complete each of the four parsimonious STEM 
momentum metrics in their first year, disaggregated by race/ethnicity and gender. 
Although students who gain early STEM momentum (i.e., complete one of the 
parsimonious STEM momentum metrics) across race/ethnicity and gender subgroups 
have higher STEM bachelor’s completion rates compared to students who do not gain 
early STEM momentum, equity gaps by race/ethnicity and gender are present—both in 
early STEM momentum rates and in STEM bachelor’s completion rates even among 
students who do gain early STEM momentum. In general, male and Asian students have 
higher STEM bachelor’s degree completion rates than their peers, a pattern that is 
apparent across the four metrics. For example, among community college students in 
State A who complete a STE pathway course in their first year, 27% of men eventually 


earn a STEM bachelor’s degree, compared to 20% of women. 
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Table 8 
Six-Year STEM Bachelor’s Degree Completion Rates, 
by First-Year STEM Coursetaking and Student Characteristics 


Completed STE pathway credits in Y1 Completed STE foundation credits in Y1 


State A, State A, StateB, StateC, | StateA, State A, StateB, StateC, 
cc NSFY cc cc cc NSFY cc cc 
All students 24% 18% 6% 25% 7% 9% 3% 9% 
Gender 
Male 27% 22% 9% 29% 12% 15% 4% 13% 
Female 20% 15% 4% 19% 3% 4% 1% 6% 
Race/ethnicity 
Asian 30% 30% 9% 31% 14% 19% 4% 14% 
Black 16% 19% 5% 14% 4% 7% <1% 3% 
rite Pune 17% 30% <1% 7% 4% <1%  —<1% 2% 
Hispanic 22% 14% 8% 28% 8% 4% 2% 3% 
Pacific Islander 14% 20% 13% <1% 25% <1% 
White 25% 18% 6% 24% 7% 8% 3% 9% 
Other/unknown 15% 15% 3% 21% 4% 15% <1% 8% 
Completed calculus credits in Y1 Completed calculus foundation credits in Y1 
State A, State A, StateB, StateC, | StateA, State A, StateB, StateC, 
cc NSFY cc cc cc NSFY cc cc 
All students 26% 23% 14% 23% 16% 12% 4% 11% 
Gender 
Male 27% 26% 15% 25% 16% 15% 6% 13% 
Female 25% 18% 10% 20% 15% 9% 3% 9% 
Race/ethnicity 
Asian 32% 27% 15% 22% 19% 16% 6% 11% 
Black 23% 29% 7% 23% 13% 15% 3% 7% 
oe nuencan) 38% 50% <1% 4% 11% 14% <1% 6% 
Hispanic 29% 24% 15% 21% 19% 5% 6% 12% 
Pacific Islander 33% 25% 24% < 1% 25% 6% 
White 27% 23% 14% 25% 16% 12% 4% 12% 
Other/unknown 10% 17% 21% 24% 9% 8% 4% 11% 
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3.4 Practical Meaning of Momentum Metrics 

Our third research question delves into the practical meaning of our preferred 
metrics by investigating the extent to which these metrics reflect students’ intent to study 
STEM versus their success within STEM, as well as by exploring the extent to which 
colleges help students gain STEM transfer momentum. 

Signaling intent, capturing momentum, or both? Thus far, our findings suggest 
that early completion of key STEM courses specified on statewide community college 
STEM transfer pathways is a robust and reliable indicator of subsequent STEM transfer 
success. While it could be that completion of these courses indicates STEM momentum 
among students aspiring to transfer into STEM majors at four-year institutions, it may 
also be that completion of these courses simply indicates which students intend to 
transfer into STEM majors at four-year institutions. To help understand the extent to 
which our momentum metrics actually measure intent for STEM transfer, we first 
examine the predictive power of our preferred STEM momentum indicators if we require 
only that students attempt (rather than complete) these courses in their first year. We then 
test our STEM momentum indicators among subsamples of students who almost certainly 
do intend to pursue a STEM bachelor’s degree. 

First, we re-run the models in Table 5, replacing each of our four parsimonious 
STEM indicators with an attempt of each relevant course rather than a completion of the 
course. If the four indicators were merely capturing STEM transfer intent, we would 
expect to observe as strong of a relationship between attempting these courses and 
earning a STEM bachelor’s degree as for completing these courses and earning a STEM 
bachelor’s degree. When re-running our results using course attempts, we find that 
individual coefficient predictors attenuate but remain positive and statistically significant. 
These findings suggest that the STEM momentum metrics may indeed capture intent but 
also capture student STEM momentum above and beyond their role as a proxy of STEM 
transfer intent. 

Second, we work to further disentangle intent from momentum by testing our 


STEM momentum indicators among subsamples of students who almost certainly intend 
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to pursue a STEM bachelor’s degree.’ In State A, our dataset includes students who enter 
a public university as a freshman, where students declare a specific major such as 
chemistry, biology, or STEM exploration. We limit this analysis to students at 
nonselective and middle-selective public universities, excluding those that are more 
selective, in order to focus on students who are more comparable to community college 
populations. In Table 9, we re-run the models from Table 7 for the nonselective and 
middle-selective four-year (MSFY) entrants in State A, with the addition of an indicator 
for whether students ever declared a STEM major as a predictor and the inclusion of 
general academic momentum, demographic, and institution-by-cohort fixed effect 
controls. The STEM momentum indicators, particularly completion of a calculus or STE 
pathway course, retain their significance as predictors of STEM bachelor’s completion 
even after controlling for whether students ever declared a STEM major (which 


unsurprisingly is also a strong predictor). 


Table 9 
State A, Nonselective and Middle-Selective Four-Year College Entrants: 
Estimates From Predictive Models of First-Year Coursetaking on 
Six-Year STEM Bachelor’s Degree Completion 


NSFY entrants MSFY entrants 
Variable Model 1 Model 2 Model 3 Model 1 Model 2 Model 3 
: .030*** .017*** .071*** .028*** 
Completed calculus credits (.002) (.002) (.002) (.001) 
Completed calculus foundation .018*** .002 .049*** .005** 
credits (.002) (.001) (.002) (.001) 
; .036*** .010*** .118*** .021*** 
Completed STE pathway credits (.002) (.003) (.002) (001) 
Completed STE foundation .010*** .004 .066*** .015*** 
credits (.002) (.002) (.002) (.002) 
: .105*** .094*** .317*** .286*** 
Declared a STEM major (.003) (.003) (.004) (.004) 
Pseudo R2 .282 .469 484 316 546 558 
Observations 56,138 56,138 56,138 127,394 127,394 127,394 


Note. Standard errors in parentheses. All models include academic and demographic controls, as well as institution- 
by-cohort fixed effects. 


**D < 001. **p <.01. *p<.05. 


3 In addition to analyses involving four-year college entrants, we also re-run the models in Tables 5 and 7 in 
all three states using subsamples of transfer-intending community college entrants who signal potential intent 
to complete a STEM bachelor’s degree by attempting a calculus foundation course (precalculus, trigonometry, 
or college algebra) in their first year. Results show that the four STEM momentum indicators, notably 
completion of a calculus or STE pathway course, remain strong predictors and have even larger individual 
coefficients predicting longer-term STEM bachelor’s completion than those shown in Table 7. 
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In order to further explore how these findings may apply to STEM-intending 
students in the community college population, we employ propensity score matching to 
select a subsample of community college peers who are similar to nonselective and 
middle-selective four-year STEM majors (in terms of available precollege variables in 
our dataset, including demographic characteristics and precollege dual enrollment 
coursetaking, such as whether the student took a math or science course as part of their 
dual enrollment experience). We use Mahalanobis matching to adjust for pretreatment 
observable differences and test for overlap between treated (started at a nonselective or 
middle-selective four-year college) and control (started at a community college) groups. 
The results suggest strong balance and common support,‘ reinforcing our identification of 
STEM majors at nonselective or middle-selective four-year colleges as reasonable 
comparison groups for the matched community college samples. As shown in Table 10, 
we re-run a logistic regression model of STEM bachelor’s completion in six years on our 
parsimonious STEM momentum indicators as well as general academic momentum, 
demographic, and institution-by-cohort fixed effect controls; however, in this instance, 
we limit our sample to community college students who match to nonselective and 
middle-selective four-year STEM majors in our propensity score analysis. Table 10 
shows that coefficients for completion of calculus and STE pathway courses remain 
positive among those community college students who are matched with nonselective 


and middle-selective four-year STEM majors.° 


4 None of the covariates in our model have a matched standardized difference above .05 in absolute value, 
and almost all of our covariates have a ratio of standard deviations between 0.95 and 1.05. 

5 In additional checks, we re-run the propensity score analysis while dropping treated observations with a 
higher or lower propensity score than the minimum or maximum propensity score of the control group and 
also limit the sample of matched community college students to those who attempted a calculus foundation 
course in their first year. In both types of checks, the four STEM momentum indicators, particularly 
completion of a calculus or STE pathway course, remain significant predictors of STEM bachelor’s degree 
attainment. 
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Table 10 
Regression Estimates on Six-Year STEM Bachelor’s Degree Completion 
for Matched Community College Students 


Calculus STE STE 

Matsnicd subercue Kaleulis foundation pathway foundation 
State A community college entrants matched to 017 017 o31** 
peers who declare STEM majors at nonselective .013 

para (.014) (.014) (.011) 
four-year institutions (.016) 
State A community college entrants matched to 
peers who declare STEM majors at middle- .056** -.013 .085*** .001 
selective four-year institutions (.020) (.017) (.015) (.020) 


Note. Standard errors in parentheses. All models control for general academic momentum, other STEM momentum 
metrics, and demographic variable blocks as well as institution-by-cohort fixed effects. The sample for these 
regressions is limited to STEM-intending community college students in State A who match with similar nonselective 
and middle-selective four-year STEM majors. 


* < 0.001. **p < 0.01. *p < 0.05. 


These results suggest that the four STEM momentum metrics do capture an 
element of STEM bachelor’s degree intent and also capture the extent to which STEM- 
intending students are progressing along the pathway toward that destination. The 
examination of four-year college students in State A also allows us to revisit descriptive 
results from Table 4 with more information about STEM intent. Table 11 presents the 
four STEM momentum metrics for nonselective and middle-selective four-year college 
entrants, disaggregated by STEM major intent, as well as for their matched peers at 
community colleges. Columns 2 and 4 of Table 11 confirm our intuition that four-year 
STEM majors are much more likely than four-year non-STEM majors to complete any of 
the four STEM indicators in their first year. For example, among nonselective four-year 
entrants, 15% of STEM majors complete a calculus course in their first year, compared to 
3% of non-STEM majors. 

Column 1| of Table 11 shows STEM momentum metrics for the 7% of community 
college students who are similar to nonselective four-year college STEM majors in terms 
of our limited array of precollege characteristics, including dual enrollment math and 
science coursetaking. The propensity score matching algorithm’s suggestion that these 
community college students may be STEM-intending is bolstered by these students’ 
levels of STE foundation and pathway coursetaking, which are substantially higher than 


the overall levels shown in Table 4. Indeed, these students’ STEM momentum metrics 
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are similar to or higher than those of their nonselective four-year STEM-intending peers, 
implying that for STEM-intending students in State A, community colleges may support 
initial STEM coursetaking just as well as (or even better than) nonselective four-year 
colleges. 

However, the story is quite different for the 12% of community college students 
matched with middle-selective four-year college STEM majors (column 3). Compared to 
their peers in STEM majors at middle-selective four-year institutions (column 4), these 
community college students are just as or more likely to complete a calculus foundation 
course; however, they are much less likely to complete a calculus course or any STE 


foundation or pathway courses. 


Table 11 
State A, Public Two- and Four-Year College Entrants: First-Year STEM Coursetaking 
(1) (2) (3) (4) 
cc NSFY cc MSFY 
(N = 92,679) (N = 56,577) (N = 92,679) (N = 127,407) 
Matched to Non- Matched to Non- 
NSFY STEM STEM STEM MSFY STEM sre STEM 
° majors : . majors : 
majors majors majors majors 
Proportion of sample 7% 18% 82% 12% 25% 75% 
Among this subsample 
completed calculus 20% 15% 3% 30% 40% 11% 
credits 
Completed calculus 40% 36% 19% 35% 32% 19% 
foundation credits 
polmpletedele 30% 24% 5% 40% 53% 10% 
pathway credits 
completsd iE 7% 6% 4% 10% 21% 3% 


foundation credits 


How well are colleges helping students gain STEM momentum? In order to 


explore how well community colleges are helping students gain STEM momentum, we 


describe STEM momentum metrics across demographic groups, states, and individual 


colleges. Results presented earlier in this paper (see Table 4) suggest that large 


percentages of students are completing likely transferable STEM coursework, but 


relatively few are completing the specific courses laid out on their state’s STEM transfer 
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pathways—that is, the courses that are most predictive of successful STEM transfer and 
bachelor’s completion. 

To examine momentum metrics across types of students, Table 12 disaggregates 
the four parsimonious STEM momentum metrics by gender and race/ethnicity, revealing 
that gaps between demographic groups already appear by the first year of community 
college. For example, Table 12 shows that Asian students are substantially more likely 
than other racial/ethnic groups to complete STE pathway, calculus, and calculus 
foundation courses in their first year at community college. Disparities across gender are 
less pronounced, although male students complete STE pathway, calculus, and calculus 


foundation courses at slightly higher rates across states compared to female students. 
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Table 12 
First-Year STEM Coursetaking by Student Gender and Race/Ethnicity 


Completed STE pathway credits in Y1 Completed STE oe credits in 
State State A, State State State phate State B, State 
A, CC NSFY B, CC c, CC A, CC NSFY cc c, CC 
All students 6% 9% 6% 4% 5% 4% 2% 6% 
Gender 
Male 6% 10% 7% 4% 4% 4% 2% 5% 
Female 5% 8% 5% 3% 6% 5% 2% 6% 
Race/ethnicity 
Asian 14% 25% 9% 7% 7% 5% 4% 8% 
Black 2% 4% 2% 2% 3% 4% 1% 5% 
oe ca 3% 5% 7% 2% 5% 5% 1% 4% 
Hispanic 5% 10% 6% 2% 4% 5% 2% 3% 
Pacific Islander 6% 8% 7% 2% 4% 6% 1% 5% 
White 7% 9% 7% 3% 6% 5% 3% 6% 
Other/unknown 4% 6% 3% 3% 3% 3% 1% 5% 
Completed calculus credits in Y1 Completed calculus ay credits in 
State State A, State State State pte State B, State 
A, CC NSFY B, CC c, cc A, CC NSFY cc c, cc 
All students 4% 5% 1% 6% 4% 9% 10% 9% 
Gender 
Male 6% 7% 2% 7% 5% 9% 11% 11% 
Female 3% 4% 1% 6% 3% 6% 8% 9% 
Race/ethnicity 
Asian 12% 19% 6% 16% 9% 12% 19% 17% 
Black 1% 2% 1% 3% 2% 6% 4% 6% 
ee AML caN 2% 3% 1% 2% 2% 7% 10% 5% 
Hispanic 3% 5% 3% 2% 3% 6% 12% 5% 
Pacific Islander 3% 6% <1% 2% 2% 7% 9% 8% 
White 5% 6% 2% 4% 5% 8% 11% 9% 
Other/unknown 4% 7% 1% 3% 3% 5% 7% 8% 
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Table 12 also suggests that the three states vary slightly in terms of the proportion 
of community college students who achieve each type of metric. In terms of STE courses, 
community colleges in States A and B seem to push a higher proportion of students to 
quickly enter transfer-level STE courses laid out on state pathways, while community 
colleges in State C seem to push the completion of STE foundation courses. Within each 
state, however, individual colleges’ performance on each metric varies widely (see Figure 
3, Panel A). In these box and whisker plots, boxes represent the interquartile range (from 
25" to 75" percentile), the line in the middle displays the median value, and the whiskers 
show the range of values adjacent to the outside values (shown as dots). Panel A displays 
each institution’s overall performance on each metric, while the remaining panels 
examine institutional performance through an equity lens, with Panel B displaying gaps 
by gender (male-female) and Panels C and D displaying gaps by race/ethnicity (White- 
Black and White-Hispanic, respectively). 

In general, Figure 3 reveals a strong degree of variation across colleges within a 
given state. For example, in State C, only 6% of community college students complete a 
calculus course in their first year, yet at a handful of colleges in the state, 15—20% of 
students do so. More importantly from an equity perspective, some colleges show 
startlingly large gaps in STE pathway or calculus course completion by gender or 
race/ethnicity. Yet other colleges show no gaps or even negative gaps (i.e., stronger 
results for women, or for Black or Hispanic students). The differences by college 
observed in overall rates of STE pathway and calculus course completion, as well as the 
variation in disparities between student groups, point to a need for more investigation into 
policies and practices at individual colleges to understand why some are more effective at 


helping students gain early STEM momentum than others. 
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Figure 3 


Institutional Variation in First-Year STEM Course Completion Rates and Gaps by Student Gender and Race/Ethnicity 


Panel A Institutional Variation: Rates of First-Year STEM Course Completion Panel B 
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Note. Figures present the distribution of first-year course completion rates for each college within a given state. Top left panel (A) shows rates for all students, and the three 
other panels (B, C, and D) display percentage point gaps, calculated by subtracting rates for female, Black, and Hispanic students from rates for male and White students, 


respectively. 
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4. Discussion 


This paper uses transcript and student outcome data on cohorts of community 
college entrants in three states to identify, validate, and describe metrics that are useful 
for college leaders looking to formatively assess efforts to improve STEM program 
outcomes, with a particular focus on students underrepresented in STEM, as part of 
broader guided pathways reforms. In general, our results affirm previous findings on the 
relationship between general early momentum measures and college completion 
(Attewell & Monaghan, 2016; Belfield et al., 2016, 2019; Wang, 2017). However, these 
general early momentum measures are less useful predictors of STEM transfer and 
bachelor’s degree completion than STEM-specific measures of early momentum. 

Across all the indicators we examine, first-year completion of calculus or other 
specific types of STEM courses—those designated by each state as required to achieve 
junior standing within a four-year STEM major—is an especially strong and reliable 
early indicator of STEM transfer and bachelor’s completion. In State A, for example, 
only 3% of all transfer-intending community college students completed a STEM 
bachelor’s degree within six years; but among those who completed a STE pathway 
course in their first year, 24% completed a STEM bachelor’s degree. The strength of 
these indicators is consistent across demographic groups, as well as across a wide variety 
of contexts (including cohorts, types of institutions, and states). However, few students 
take these courses in their first year, and there are widespread equity gaps by 


race/ethnicity and gender. 


4.1 Scaffolding Foundational and Transfer Pathway Courses 

Our results suggest that completing any of four specific types of STEM courses in 
the first year of community college predicts eventual bachelor’s degree attainment in 
STEM: (1) calculus courses, which consistently appear on statewide STEM transfer 
pathways, (2) the courses that serve as a foundation for calculus (e.g., Precalculus, 
Trigonometry), (3) STE pathway courses that appear on statewide transfer pathways for 
specific majors (e.g., Chemistry I), and (4) STE foundation prerequisites for those 


courses (e.g., Introduction to Chemistry). However, the predictive power of calculus and 
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STE pathway courses is substantially stronger and more consistent than that of their 
prerequisite foundation courses. 

Foundational STEM courses often consist of coursework that could be completed 
at the high school level, such as trigonometry or a full-year sequence of introductory 
chemistry, but are inconsistently required of high school students to earn a diploma and 
may not be offered by all high schools. Community colleges receive a steady stream of 
STEM-intending students who have either not recently completed high school or not 
completed such foundational coursework in high school. As a result, colleges typically 
place these students into a series of preparatory courses. For example, in two of the states 
under study, prerequisites to transfer-level biology and chemistry include at least three 
separate preparatory subject courses. Although these courses do earn students college- 
level credit, in many ways the foundational STEM track is analogous to the traditional 
developmental math and English system—a system that a large body of research finds to 
be ineffective at helping students who are underprepared in these subjects move forward 
toward graduation, with disproportionate effects on low-income students and students of 
color (Chen & Simone, 2016; Jaggars & Bickerstaff, 2018). 

With this context in mind, STEM departments such as physics, chemistry, or 
biology may find it helpful to learn about college mathematics reforms that accelerate the 
academic momentum of students who arrive at college underprepared in math. These 
reforms reduce or eliminate prerequisite developmental math coursework and allow 
students to immediately enroll in more challenging math courses such as Statistics or 
College-Level Algebra, while providing learning supports that are tailored specifically to 
helping students gain and practice the skills needed to be successful in the course 
(Braithwaite et al., 2020). Successful acceleration efforts typically include both curricular 
and instructional reform. From a curricular perspective, prerequisite sequences can be 
shortened by removing content that is repetitive or unnecessary for success in the 
subsequent course or can be redesigned into corequisite courses that provide just-in-time 
instruction and practice for challenging concepts. From an instructional perspective, 
students can outperform expectations in math courses that focus on student collaboration, 
active student thinking and discussion, the grounding of problems in real-world contexts 


to develop conceptual understanding, and explicit attention to students’ organizational 
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and study habits (Bickerstaff & Edgecombe, 2019; Zachry Rutschow et al., 2019; Wang 
et al., 2021). 

In general, developmental reform efforts do not focus on STEM-intending 
students; most reforms in developmental mathematics aim to help non-STEM-intending 
students fulfill degree requirements in quantitative reasoning, statistics, or college-level 
algebra (Brathwaite et al., 2020). For students bound for calculus and calculus-heavy 
STEM courses, only one reform effort has been well documented: the Dana Center 
Mathematics Pathway (DCMP), which has been deployed across multiple community 
colleges in Texas (Zachry Rutschow et al., 2019). For STEM-intending students, DCMP 
colleges teach a two-course Reasoning with Functions sequence that replaces the 
traditional College-Level Algebra and Precalculus sequence. Although the Functions 
approach does not shorten the prerequisite sequence for calculus, its curriculum and 
instruction are tailored to helping students master and become confident in the 
mathematics concepts and skills needed in STEM disciplines; for example, students use 
functions to model and solve meaningful problems in science, technology, and 
engineering. ° 

In addition to rethinking their prerequisite sequences, STEM departments can also 
encourage and assist underserved students to take College-Level Algebra and other 
foundational STEM coursework in high school by offering these courses through dual 
enrollment programs. For example, one rigorous study in Florida found that among high 
school students who were at the margin of eligibility for dual enrollment College-Level 
Algebra, taking the course improved the likelihood that they would enroll in college and 
enter a STEM major; these findings were particularly strong for Black and Hispanic 
students (Minaya, 2021). 

Finally, in order to connect incoming students to appropriate coursework and 
supports, community colleges need to identify students who are interested in STEM 
majors early on. As part of the guided pathways reform framework, many community 
colleges have reorganized individual programs into broad fields of study, or “meta- 


majors” (e.g., business, health, or STEM), which has enabled reforms of traditional new 


® For more detailed information about the DCMP Reasoning with Functions courses, see Dana Center 
Mathematics Pathways (n.d.-a, n.d.-b). 
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student orientation to better identify students who, for example, aspire to transfer into a 
STEM program at a four-year institution. By identifying in students’ first year which 
broad fields of study they intend to pursue, community colleges are better equipped to 
connect them to courses, faculty mentors, and other experiences to help build early 


program momentum (Jenkins et al., 2020). 


4.2 Increasing Equity of Access for State Transfer Pathways 

Community colleges seeking equitable access and outcomes for their students 
should consider what other systemic challenges might contribute to low and inequitable 
rates of transfer-level STEM course completion and how the implementation of state 
transfer pathways can support success for all students. While the positive relationship 
between early STEM coursework and longer-term STEM transfer outcomes is fairly 
consistent across our results, much work remains to ensure access to and adequate 
support for all college students who wish to attempt STEM coursework, no matter where 
they begin their college journey. Table 12 demonstrates the stark disparities in first-year 
STEM coursetaking between student racial/ethnic groups; Asian and White students are 
more likely in all states to complete a STE pathway course in their first year at 
community college than Black, Hispanic, and Native American students. It also 
demonstrates how these disparities can compound over time; within the context of a 
prerequisite sequence for science and math courses, Asian and White students are also 
more likely to earn credits in foundational math and STE courses in their first year, 
increasing the rate of their momentum toward transfer and four-year degree completion 
over other student groups. 

An additional consideration for colleges seeking to improve access to and success 
in state STEM transfer pathways is the availability of calculus, calculus foundation, STE 
pathway, and STE foundation courses each term. A supplementary analysis of the 
availability of STE pathway and foundation courses in our sample finds that most courses 
offered by term in key STE subject areas (e.g., biology, chemistry, and physics) are not 
the courses specified on statewide STEM transfer pathways. For example, in State B, 
STE pathway courses account for 24% of the biology courses offered at community 
colleges in fall 2018, and STE foundation courses account for only 12% of the biology 


courses offered. A similar pattern emerges in State C, where calculus courses comprise 
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16% of college-level math courses offered by community colleges in fall 2019, and 
calculus foundation courses account for 21% of college-level math courses. These 

supplementary analyses raise further questions, such as: What are the other types of 
STEM courses that students are taking, and toward what programs do those courses 


count? 


4.3 Identifying Areas for Future Research 

Our analyses raise a number of additional questions for future research. First, the 
college-by-college variation in rates of early STEM momentum raises further questions 
around why some colleges are more effective at helping students gain early STEM 
momentum. What can be learned from colleges that are doing better in this area, 
particularly from colleges that are more effective in increasing STEM momentum for 
women and students of color? Second, our analysis treats course types (e.g., STE 
pathway) as uniform, but within each type of course, there is surely variation in how 
students experience the course content. Future research can examine how much of the 
predictive power of early STEM coursework is due to its specific topic and how much is 
due to other factors related to the classroom dynamic, such as the instructor or course 
composition. For example, within a specific course such as Chemistry I, to what extent 
do sections of the course vary in terms of how well women and students of color succeed 
in (and beyond) the course? 

Finally, each state in our study has developed transfer pathways in other areas, 
such as business, nursing, or education. Would our methodological framework— 
identifying courses specified on the given state’s transfer pathway—work equally as well 
in operationalizing program momentum in other disciplines? In particular, our analysis 
finds that both the key STEM math courses (calculus) as well as key non-math STEM 
courses (STE pathway courses) are similar in their relationship to longer-term STEM 
outcomes. Perhaps any program’s momentum involves two elements—program-relevant 
math and other (non-math) program-relevant content. Or perhaps in other subjects, 
general but highly important content courses in areas such as English or history might 
play a similar role to math within STEM. In general, extensions of the program 
momentum framework can examine the relative importance of program-specific courses 


versus program-critical (but not program-specific) courses such as math and English. 
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5. Conclusion 


In this study, we examine postsecondary college transcript and degree records 
from hundreds of thousands of college students across dozens of colleges in three states 
to explore and test metrics that may be useful for community college leaders to 
formatively assess their college’s efforts to improve STEM transfer outcomes. We find 
that a relatively simple set of STEM momentum metrics—notably early completion of 
calculus or STE coursework included in statewide STEM transfer pathways and, to a 
lesser degree, the prerequisites to these courses—are reliable indicators of subsequent 
STEM transfer success across the wide-ranging state and institutional contexts observed 
in this study. We also find that these metrics are robust predictors among subgroups of 
students by race/ethnicity and gender. Yet community college students have relatively 
low rates of completion of these key STEM courses, and disparities in completion of 
these courses by race/ethnicity and gender are common. Low and inequitable rates of 
STEM transfer and bachelor’s completion can be traced back to low and inequitable rates 
of early STEM momentum. Some colleges are more effective than others in correcting 
for these inequities and in helping more students gain early STEM momentum, and the 
variation we observe across institutions implies that improvement on these key early 
indicators is indeed possible. The STEM momentum metrics identified in this paper offer 
promising utility in the formative assessment of community college reforms aimed at 
strengthening STEM transfer outcomes and closing equity gaps in STEM bachelor’s 


attainment. 
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Appendix 


Table Al 


Logit Model Coefficients on STEM Bachelor’s Degree Completion, 
Including Interaction Terms for Historically Underrepresented Groups 


STEM 
Uadaniae Momen- State A, State A, State B, State C, 
tum NSFY cc cc cc 
Metric 
Completed 6+ credits in term 1 0.012** 0.002 -0.000 -0.001 
(0.005) (0.002) (0.002) (0.002) 
Completed 12+ credits in Term 1 0.001 0.002 0.001 -0.011*** 
0.012** (0.002) (0.002) (0.002) 
Completed 12+ credits in Y1 0.033*** 0.024*** 0.003 0.008*** 
(0.007) (0.003) (0.002) (0.003) 
Completed 24+ credits in Y1 0.026*** 0.012*** 0.001 0.003 
(0.003) (0.002) (0.002) (0.002) 
Completed CL math credits in Y1 0.015*** 0.001 0.017*** 
(0.002) (0.001) (0.002) 
Completed CL English credits in Y1 -0.007*** -0.008*** 0.003 -0.001 
(0.002) (0.002) (0.002) (0.002) 
Persisted from term 1 to term 2 -0.007 -0.003 0.002 -0.024*** 
(0.007) (0.003) (0.003) (0.003) 
GPA of 3.0+ in Y1 0.019*** 0.018*** 0.009*** 0.036*** 
(0.002) (0.002) (0.001) (0.002) 
Completed STE pathway credits Yr1 xX 0.030*** 0.034*** 0.010*** 0.039*** 
(0.002) (0.002) (0.001) (0.002) 
Completed STE foundation credits in Y1 xX 0.011*** 0.001 0.006*** 0.007*** 
(0.003) (0.002) (0.002) (0.002) 
Completed calculus credits in Y1 xX 0.029*** 0.021*** 0.009*** 0.034*** 
(0.002) (0.002) (0.001) (0.002) 
Completed calculus foundation credits in Y1 X 0.018*** -0.002 0.005*** 0.015*** 
(0.002) (0.002) (0.001) (0.002) 
Age -0.001*** -0.000** -0.000 -0.001*** 
(0.000) (0.000) (0.000) (0.000) 
Female -0.002 -0.000 -0.002 -0.022*** 
(0.008) (0.004) (0.004) (0.003) 
HU_Completed 6+ credits in term 1 0.008 -0.010* 0.006* -0.002 
(0.013) (0.005) (0.004) (0.006) 
HU_Completed 12+ credits in term 1 -0.006 0.007 0.001 -0.019** 
(0.007) (0.005) (0.003) (0.008) 
HU_Completed 12+ credits in Y1 0.010 -0.008 -0.004 -0.008 
(0.017) (0.007) (0.004) (0.011) 
HU_Completed 24+ credits in Y1 -0.007 0.007 -0.003 0.015** 
(0.008) (0.005) (0.003) (0.007) 
HU_Completed CL math credits in Y1 0.006 0.014** 0.002 0.018* 
(0.008) (0.007) (0.004) (0.010) 


af 


STEM 


Variables Momen- State A, State A, State B, State C, 
tum NSFY ce cc eC 
Metric 
HU_Completed CL English credits in Y1 0.004 0.002 -0.003 0.007 
(0.008) (0.004) (0.003) (0.006) 
HU_Persisted from term 1 to term 2 -0.012 -0.021*** -0.003 0.003 
(0.015) (0.006) (0.004) (0.010) 
HU_GPA of 3.0+ in Y1 0.005 -0.001 -0.001 -0.011* 
(0.008) (0.005) (0.003) (0.006) 
HU_Completed calculus credits in Y1 xX 0.007 0.004 0.000 -0.000 
(0.007) (0.005) (0.003) (0.008) 
HU_Completed calculus foundation credits in Y1 X 0.006 -0.002 0.005 -0.008 
(0.006) (0.006) (0.003) (0.008) 
HU_Completed statistics credits in Y1 0.010 -0.005 -0.004 -0.010 
(0.011) (0.007) (0.004) (0.010) 
HU_Completed other CL math credits in Y1 -0.006 -0.014** 0.001 -0.030** 
(0.010) (0.006) (0.004) (0.012) 
HU_Completed dev. ed. math credits in Y1 0.006 -0.005 0.004 -0.006 
(0.008) (0.005) (0.004) (0.006) 
HU_Math GPA of 3.0+ in Y1 0.007 -0.003 0.001 0.012* 
(0.007) (0.005) (0.003) (0.008) 
HU_Completed STE pathway credits in Y1 X 0.005 -0.008* -0.005 -0.010 
(0.007) (0.005) (0.003) (0.009) 
HU_Completed STE foundation credits in Y1 X -0.002 0.009 -0.009 -0.026*** 
(0.009) (0.006) (0.008) (0.010) 
HU_Cd. other likely transferable STE credits in Y1 0.037 0.032** 0.005 0.013 
(0.023) (0.013) (0.010) (0.009) 
HU_Completed other likely CTE STE credits in Y1 0.043*** 0.027*** -0.000 -0.054** 
(0.015) (0.008) (0.006) (0.025) 
HU_Completed any STE credits in Y1 -0.028 -0.018 -0.001 0.009 
(0.024) (0.014) (0.010) (0.010) 
HU_STE GPA of 3.0+ in Y1 0.000 0.002 0.003 0.007 
(0.007) (0.005) (0.003) (0.007) 
FEM_Completed 6+ credits in term 1 -0.005 0.006 0.000 -0.001 
(0.008) (0.004) (0.003) (0.003) 
FEM_Completed 12+ credits in term 1 -0.001 -0.006** 0.002 0.004 
(0.004) (0.003) (0.003) (0.004) 
FEM_Completed 12+ credits in Y1 -0.015 -0.009 0.001 -0.003 
(0.011) (0.006) (0.004) (0.005) 
FEM_Completed 24+ credits in Y1 -0.001 0.006* -0.001 -0.009*** 
(0.005) (0.004) (0.003) (0.004) 
FEM_Completed CL math credits in Y1 0.005 -0.009** 0.001 0.007 
(0.004) (0.004) (0.003) (0.005) 
FEM_Completed CL English credits in Y1 0.001 -0.006** -0.005* 0.002 
(0.004) (0.003) (0.003) (0.003) 
FEM_Persisted from term 1 to term 2 -0.018* -0.021*** -0.005 -0.001 
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vananlee Momen- State A, State A, State B, State C, 
tum NSFY ce cc eC 
Metric 
(0.010) (0.005) (0.004) (0.005) 
FEM_GPA of 3.0+ in Y1 -0.008** -0.010*** -0.000 -0.009*** 
(0.004) (0.003) (0.002) (0.003) 
FEM_Completed calculus credits in Y1 X 0.001 0.010*** -0.000 0.003 
(0.004) (0.003) (0.002) (0.004) 
FEM_Completed calculus fdn. credits in Y1 xX -0.003 0.008** -0.001 -0.001 
(0.004) (0.004) (0.002) (0.004) 
FEM_Completed statistics credits in Y1 -0.003 -0.007* -0.001 -0.016*** 
(0.004) (0.004) (0.002) (0.004) 
FEM_Completed other CL math credits in Y1 -0.001 0.008** -0.002 -0.028*** 
(0.004) (0.003) (0.002) (0.005) 
FEM_Completed dev. ed. math credits in Y1 -0.008** -0.011*** -0.004 0.003 
(0.004) (0.003) (0.003) (0.003) 
FEM_Math GPA of 3.0+ in Y1 0.000 -0.002 0.003 0.005 
(0.003) (0.003) (0.002) (0.003) 
FEM_Completed STE pathway credits in Y1 X 0.009*** 0.009*** -0.000 0.002 
(0.003) (0.003) (0.002) (0.004) 
FEM_Completed STE fdn. credits in Y1 X -0.008 -0.004 -0.005 -0.004 
(0.005) (0.004) (0.003) (0.004) 
FEM_Cd. other likely transferable STE credits in Y1 0.013 0.021 0.082 0.006* 
(0.019) (0.013) (3.374) (0.003) 
FEM_Cd. other likely CTE STE credits in Y1 0.022** 0.034*** 0.008 0.007 
(0.010) (0.007) (0.007) (0.011) 
FEM_Completed any STE credits in Y1 0.009 -0.004 -0.078 0.013*** 
(0.020) (0.013) (3.374) (0.004) 
FEM_STE GPA of 3.0+ in Y1 0.005 -0.002 -0.001 0.002 
(0.003) (0.003) (0.002) (0.003) 
Observations 50,230 84,881 50,594 94,875 


Note. Standard errors in parentheses. All models include institution-by-cohort fixed effects. Variables with interaction 


terms are given the following prefixes: Interaction effects for women are labeled “FEM” and for historically 


underrepresented students of color—including Black, Hispanic, Native American, and Pacific Islander students—are 


labeled “HU.” 


*** 9D < 001, **p<.01, *p<.05 
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